Methods and apparatus for scalable metrics gathering from secure data centers

ABSTRACT

A method in an accessible datacenter connected to a data storage network is provided. The accessible datacenter comprises: at least one data pipeline, each of the at least one data pipeline connected to a respective secure datacenter and configured to receive data from each respective secure datacenter; a storage layer, configured to store the data received via the at least one data pipeline; and a visualization layer, configured to provide a user interface and to receive user input requesting access to the data. The method comprises: detecting scaling of the data storage network, the scaling comprising increasing available storage of the data storage network, the increasing available storage creating increased available storage; and providing access to the increased available storage continuously, via the visualization layer, without rendering the data storage network inaccessible during scaling; wherein the data is associated with the increased available storage.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit as a divisional of U.S. patentapplication Ser. No. 15/141,250, having the same title as thisapplication, and filed on Apr. 28, 2016. This application incorporatesthe prior application into the present application by reference.

TECHNICAL FIELD

Embodiments of the subject matter described herein relate generally tosecure datacenters. More particularly, embodiments of the subject matterrelate to providing user access to data stored in a secure datacenter.

BACKGROUND

A datacenter is a facility used to house computer systems and associatedcomponents, such as telecommunications and storage systems. It generallyincludes multiple power supplies, data communications connections,environmental controls (e.g., air conditioning, fire suppression) andvarious security devices. Large datacenters may be industrial scaleoperations which use a large amount of electricity. A datacenter may beused to store any type of data, and various levels of security andcapacity may be used for a datacenter. A data storage network mayinclude more than one datacenter. End users may require access to datastored at a datacenter and/or metrics data associated with the storeddata. However, the end user experience is usually interrupted when newstorage resources are added to a datacenter, or when a new datacenter isadded to a data storage network. In this scenario, the datacenter ordata storage network may be required to “go offline” for a period oftime while the additional storage resources are being added. In anotherscenario, a user may not have adequate security clearance to access asecure datacenter, but still require access to a subset of the datastored at a secure datacenter (e.g., non-secure data stored at a securedatacenter).

Accordingly, it is desirable to provide a seamless end user experiencewithout interruptions and “offline” time, which prevents an end userfrom accessing his data. It is also desirable to provide a mechanism fornon-secure data access. Furthermore, other desirable features andcharacteristics will become apparent from the subsequent detaileddescription and the appended claims, taken in conjunction with theaccompanying drawings and the foregoing technical field and background.

SUMMARY

An accessible datacenter connected to a data storage network isprovided. The accessible datacenter includes: at least one datapipeline, each of the at least one data pipeline connected to arespective secure datacenter and configured to receive data from eachrespective secure datacenter; a storage layer, configured to store thedata received via the at least one data pipeline; a visualization layer,configured to provide a user interface and to receive user inputrequesting access to the data; and at least one processor. The at leastone processor is configured to: detect scaling of the data storagenetwork, the scaling comprising increasing available storage of the datastorage network, the increasing available storage creating increasedavailable storage; and provide access to the increased available storagecontinuously, via the visualization layer, without rendering the datastorage network inaccessible during scaling; wherein the data isassociated with the increased available storage.

A method in an accessible datacenter connected to a data storage networkis provided. The accessible datacenter comprises: at least one datapipeline, each of the at least one data pipeline connected to arespective secure datacenter and configured to receive data from eachrespective secure datacenter; a storage layer, configured to store thedata received via the at least one data pipeline; and a visualizationlayer, configured to provide a user interface and to receive user inputrequesting access to the data. The method comprises: detecting scalingof the data storage network, the scaling comprising increasing availablestorage of the data storage network, the increasing available storagecreating increased available storage; and providing access to theincreased available storage continuously, via the visualization layer,without rendering the data storage network inaccessible during scaling;wherein the data is associated with the increased available storage.

An accessible datacenter connected to a data storage network isprovided. The accessible datacenter includes: at least one datapipeline, each of the at least one data pipeline connected to arespective secure datacenter and configured to receive data from eachrespective secure datacenter; a storage layer, configured to store thedata received via the at least one data pipeline; a visualization layer,configured to provide a user interface and to receive user inputrequesting access to the data; at least one processor; and non-transientcomputer readable media encoded with programming instructions. Theprogramming instructions are configurable to cause the at least oneprocessor to: detect scaling of the data storage network, the scalingcomprising increasing available storage of the data storage network, theincreasing available storage creating increased available storage; andprovide access to the increased available storage continuously, via thevisualization layer, without rendering the data storage networkinaccessible during scaling; wherein the data is associated with theincreased available storage.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the subject matter may be derived byreferring to the detailed description and claims when considered inconjunction with the following figures, wherein like reference numbersrefer to similar elements throughout the figures.

FIG. 1 is a diagram of a secure datacenter, in accordance with thedisclosed embodiments;

FIG. 2 is a diagram of an accessible datacenter, in accordance with thedisclosed embodiments;

FIGS. 3A-3B illustrate a diagram of a data network, in accordance withthe disclosed embodiments;

FIG. 4 is an embodiment of a process for scaling a data storage network;

FIG. 5 is an embodiment of a process for integrating a new securedatacenter into a data storage network; and

FIG. 6 is an embodiment of a process for providing an end user access tosecure datacenter metrics data.

DETAILED DESCRIPTION

The following detailed description is merely illustrative in nature andis not intended to limit the embodiments of the subject matter or theapplication and uses of such embodiments. As used herein, the word“exemplary” means “serving as an example, instance, or illustration.”Any implementation described herein as exemplary is not necessarily tobe construed as preferred or advantageous over other implementations.Furthermore, there is no intention to be bound by any expressed orimplied theory presented in the preceding technical field, background,brief summary or the following detailed description.

The present disclosure describes methods an apparatus for increasingdata storage resources in a data network, and providing uninterruptedend user access to the data network during this “scaling” of the datanetwork. To accomplish this, an accessible datacenter is provided on thedata network, wherein the accessible datacenter receives metrics dataassociated with the increased data storage resources. The accessibledatacenter provides a user interface (e.g., a “visualization layer”) forrequesting particular data (e.g., the metrics data) from the increaseddata storage resources, and data storage (e.g., a “storage layer”) forretaining the particular data.

In the context of the present disclosure, a “datacenter” may beimplemented as any group of servers with as restricted direct access,e.g., a cluster, a pod, a super pod, or a datacenter. A securedatacenter stores client-specific, confidential data in combination withother types of data. A non-secure, less-secure, or “accessible”datacenter does not store client-specific or other confidential data,and is thus provided as a mechanism for a user to accessnon-confidential data.

Turning now to the figures, FIG. 1 is a diagram of a secure datacenter100, in accordance with the disclosed embodiments. It should beappreciated that FIG. 1 depicts a simplified embodiment of the securedatacenter 100, and that some implementations of the secure datacenter100 may include additional elements or components. The secure datacenter100 may be implemented as a single-tier datacenter or a multi-tierdatacenter. As shown, the secure datacenter 100 includes a plurality ofservers 102, a network infrastructure 104, at least one networkreporting server 106, and a data accumulator 108. It should beappreciated that the plurality of servers 102, the networkinfrastructure 104, the at least one network reporting server 106, andthe data accumulator 108, and any corresponding logical elements,individually or in combination, are exemplary means for implementing asecure datacenter 100.

The plurality of servers 102 may be implemented using any number ofapplication servers, and each server may be implemented using anysuitable computer that includes at least one processor, some form ofcomputer memory, and input/output (I/O) communication hardware andsoftware. In some embodiments, the plurality of servers 102 includes oneor more dedicated computers. In some embodiments, the plurality ofservers 102 includes one or more computers carrying out otherfunctionality, in addition to server operations.

The network infrastructure 104 is implemented using a plurality ofactive network devices, including but not limited to: one or morerouters, firewalls, and/or switches. The active network devices are usedto transmit data inside of a datacenter, as well as to serve externalconnections to other datacenters or end users.

The at least one network reporting server 106 is a suitable computerthat includes at least one processor, some form of computer memory, andinput/output (I/O) communication hardware and software, and functions toprovide the metrics data from the network infrastructure to the dataaccumulator. The network infrastructure 104 collects data about itsworking conditions, i.e. volume of processed data, connected peers,temperature of its core components, and CPU and memory utilization.Collected information is stored internally and it is a responsibility ofthe at least one network reporting server 106 to extract the informationfrom the network infrastructure 104 devices using a protocol supportedby the network infrastructure 104 devices, and to post the data to adata accumulator 108 using a protocol supported by the data accumulator108.

The data accumulator 108 may be implemented using a queue, a buffer, orany region of a physical memory storage used to temporarily store a setof data while the set of data is being moved from one place to another.For purposes of the present disclosure, the set of data may includemetrics data obtained from the network infrastructure 104 or fromservers 102. Metrics data may include information related to performanceof the secure datacenter 100. Examples of metrics data may include,without limitation, system-level metrics data (disk, I/O, CPU, memory,load); application-specific metrics data (how long does a particularpage take to render? how long does a particular SQL query take? how manytimes has a particular person logged in successfully or beenunsuccessful?). Metrics data may be viewed by an end user to definebaselines and/or thresholds for alerts, to view metrics trends, andsystem health. Metrics data includes any accessible and retrievable setof data that includes a name of a particular measurement, the time ofthe measurement, and the result of the measurement.

FIG. 2 is a diagram of an accessible datacenter 200, in accordance withthe disclosed embodiments. As shown, the accessible datacenter 200includes data accumulators 202, a plurality of storage nodes 204, and avisualization layer 206. It should be appreciated that these componentsand any corresponding logical elements, individually or in combination,are exemplary means for implementing an accessible datacenter 200. Itshould be appreciated that FIG. 2 depicts a simplified embodiment of theaccessible datacenter 200, and that some implementations of theaccessible datacenter 200 may include additional elements or components.

The accessible datacenter 200 includes a “storage layer” at whichmetrics data is stored after it is received from a secure datacenter andis available for retrieval, by a user, via a visualization layer 206. Inthe exemplary embodiment shown, The storage layer includes at least onedata accumulator 202 associated with each secure datacenter incommunication with the accessible datacenter 200, and a plurality ofstorage nodes 204 that are communicatively coupled to the dataaccumulators 202. The data accumulators 202 may be implemented asdescribed previously with regard to FIG. 1. Each of the dataaccumulators 202 may be implemented using a queue, a buffer, or anyregion of a physical memory storage used to temporarily store a set ofdata while the set of data is being moved from one place to another. Inthis particular example, the data accumulators 202 are configured toreceive data (e.g., metrics data), and then to transmit and store thereceived data at the plurality of storage nodes 204.

The plurality of storage nodes 204 may be implemented using one or morestorage servers. Similar to the plurality of servers describedpreviously with regard to FIG. 1, the plurality of storage nodes 204 maybe implemented using any number of servers, and each server may beimplemented using any suitable computer that includes at least oneprocessor, some form of computer memory, and input/output (I/O)communication hardware and software.

The visualization layer 206 is configured for user interaction with theaccessible datacenter 200. The visualization layer 206 may be configuredto receive end user requests for metrics data, and to retrieve therequested data from the plurality of storage nodes 204. Thus, thevisualization layer 206 provides user access to a set of requested data.The visualization layer 206 may be implemented as a computer systemusing at least one processor, a system memory element, and a userinterface. The at least one processor may be implemented or performedwith one or more general purpose processors, a content addressablememory, a digital signal processor, an application specific integratedcircuit, a field programmable gate array, any suitable programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination designed to perform the functionsdescribed here. In particular, the at least one processor may berealized as one or more microprocessors, controllers, microcontrollers,or state machines. Moreover, the at least one processor may beimplemented as a combination of computing devices, e.g., a combinationof digital signal processors and microprocessors, a plurality ofmicroprocessors, one or more microprocessors in conjunction with adigital signal processor core, or any other such configuration.

The at least one processor communicates with system memory. The systemmemory may be realized using any number of devices, components, ormodules, as appropriate to the embodiment. In practice, the systemmemory could be realized as RAM memory, flash memory, EPROM memory,EEPROM memory, registers, a hard disk, a removable disk, or any otherform of storage medium known in the art. In certain embodiments, thesystem memory includes a hard disk, which may also be used to supportfunctions of the at least one processor. The system memory can becoupled to the at least one processor such that the at least oneprocessor can read information from, and write information to, thesystem memory. In the alternative, the system memory may be integral tothe at least one processor.

The user interface may include or cooperate with various features toallow a user to interact with the visualization layer 206 of theaccessible datacenter 200. Accordingly, the user interface may includevarious human-to-machine interfaces, e.g., a keypad, keys, a keyboard,buttons, switches, knobs, a touchpad, a joystick, a pointing device, avirtual writing tablet, a touch screen, a microphone, or any device,component, or function that enables the user to select options, inputinformation, or otherwise control the operation of the visualizationlayer 206 of the accessible datacenter 200. For example, the userinterface could be manipulated by an operator to make menu selectionsfor purposes of requesting metrics data applicable to a particulardatacenter that is in communication with the accessible datacenter 200.In certain embodiments, the user interface may include or cooperate withvarious features to allow a user to interact with the visualizationlayer of the accessible datacenter 200 via graphical elements renderedon a display element. Accordingly, the user interface may initiate thecreation, maintenance, and presentation of a graphical user interface(GUI). In certain embodiments, the display element implementstouch-sensitive technology for purposes of interacting with the GUI.Thus, a user can manipulate the GUI by moving a cursor symbol renderedon the display element, or by physically interacting with the displayelement itself for recognition and interpretation, via the userinterface.

Using the visualization layer 206, an end user may request a set of data(e.g., a set of metrics data) that satisfies certain user-definedcriteria. The user-defined criteria may be received at the visualizationlayer 206 via a user interface of some type (described previously). Forexample, an end user may provide user input requesting a set of metricsdata, wherein the set of data is between a beginning date/time andending at a particular date/time. User entered data may also include thename of a particular parameter of interest. Once the set of data hasbeen requested, the visualization layer 206 filters the data stored atthe storage nodes 204 using the user-provided criteria to identify therequested data, retrieves the data from the storage nodes, transforms itaccording to the user request, and then presents results to the end userin a format requested by the user (e.g., using a visual diagram or astructured data set).

FIGS. 3A-3B illustrate a diagram of a data network 300, in accordancewith the disclosed embodiments. These elements and features of the datanetwork 300 may be operatively associated with one another, coupled toone another, or otherwise configured to cooperate with one another asneeded to support the desired functionality—in particular, scalabilityof the data network 300, as described herein. For ease of illustrationand clarity, all of the various physical, electrical, and logicalcouplings and interconnections for these elements and features are notdepicted in FIGS. 3A-3B. Moreover, it should be appreciated thatembodiments of the data network 300 will include other elements,modules, and features that cooperate to support the desiredfunctionality. For simplicity, FIGS. 3A-3B only depict certain elementsthat relate to the scalability techniques described in more detailbelow.

In the exemplary embodiment shown, the data network 300 includes a datastorage network and at least one data transmission network. Here, thedata storage network includes two secure datacenters: first securedatacenter 302 and second secure datacenter 304. The data network 300further includes an accessible datacenter 306. It should be appreciatedthat the data network 300 may include any number of secure datacenters(i.e., a plurality of secure datacenters), and that the accessibledatacenter 306 may be communicatively coupled to any number of securedatacenters. It should further be appreciated that the first securedatacenter 302 and the second secure datacenter 304 depict simplifiedembodiments of secure datacenters, and that other embodiments of asecure datacenter may include additional elements or components.

Each of the first secure datacenter 302 and the second secure datacenter304 include components consistent with the description of an exemplaryembodiment of a secure datacenter described previously with regard toFIG. 1, and these components will not be redundantly described here.Further, the accessible datacenter 306 is consistent with, and includescomponents consistent with, the exemplary embodiment of an accessibledatacenter described previously with regard to FIG. 2, and thesecomponents will not be redundantly described here.

The data network 300 further includes a plurality of data pipelines 308,which function to communicatively connect a first data accumulator of asecure datacenter (e.g., first secure datacenter 302, second securedatacenter 304) to a second data accumulator of the accessibledatacenter 306. Each data pipeline represents a link between a securedatacenter 302, 304 and an accessible datacenter 306, and is an isolateddata transmission network. As described previously with regard to FIG.2, each secure datacenter includes a storage layer, which comprises atleast a data accumulator and a plurality of storage nodes. Each dataaccumulator functions as a buffer or queue for metrics data, which istransmitted via one of the plurality of data pipelines 308 to adedicated data accumulator of the accessible datacenter 306. Theaccessible datacenter 306 may include more than one data accumulator,and each data accumulator is dedicated to receive data from a particularsecure datacenter. Each of the dedicated data accumulators (of theaccessible datacenter 306) is further configured to transfer (i.e.,transmit or “shift”) received data into a plurality of storage nodes(e.g., storage servers), from which the received data is accessible byan end user via the visualization layer of the accessible datacenter306.

Each data pipeline 308 includes dedicated servers on (1) the securedatacenter 302, 304 side, (2) on the accessible datacenter 306 side, and(3) servers potentially shared with other data pipelines. In oneexemplary embodiment, one or both of the secure datacenters 302, 304includes: five dedicated servers implemented as a data buffer (e.g.,using Kafka); three dedicated servers configured to implement a clusterorchestration application (e.g., ZooKeeper); and two dedicated serversconfigured to “push” data from a secure datacenter 302, 304 to theaccessible datacenter 306 (e.g., using Mirror Maker). In this exemplaryembodiment, the accessible datacenter 306 includes: five dedicatedservers implemented as a data buffer (e.g., using Kafka); and threededicated servers configured to implement a cluster orchestrationapplication (e.g., ZooKeeper). Also, in this exemplary embodiment,servers potentially shared with other data pipelines includes twodedicated servers implemented as consumers which “shovel” data from oneor more local data buffers (e.g., those implemented using Kafka) toother storage nodes. This particular exemplary embodiment includes tenservers on the secure datacenter 302, 304 side, eight dedicated serverson the accessible datacenter 306 side, and four or more dedicatedservers that are potentially shared with other data pipelines' serverson the accessible datacenter 306 side.

The purpose of the exemplary configuration of the data network 300 shownis to provide an end user, via the visualization layer of the accessibledatacenter 306, access to metrics data associated with the first securedatacenter 302 and/or the second secure datacenter 304, withoutrequiring the user to navigate the increased security protocolsassociated with a secure datacenter, and without making the data network300 unavailable during integration of a new secure datacenter (e.g.,without taking the data network 300 offline). The accessible datacenter306 provides a decreased level of security, and seamless, uninterruptedend user accessibility of metrics data associated with a securedatacenter of the data network 300.

Here, the data network 300 includes two components: (1) a data storagenetwork that includes all storage nodes of the first secure datacenter302, the second secure datacenter 304, and the accessible datacenter306; and (2) a data transmission network (e.g., two data accumulatorsand a data pipeline connection between them) for each link between asecure datacenter and the accessible datacenter 306. Each link between asecure datacenter and the accessible datacenter 306 is an isolated datatransmission network. Further, the data storage network is divided intoisolated sub-networks, and each of the sub-networks is connected withone or more data transmission networks.

FIG. 4 is an embodiment of a process 400 for scaling a data storagenetwork. The various tasks performed in connection with process 400 maybe performed by software, hardware, firmware, or any combinationthereof. For illustrative purposes, the following description of process400 may refer to elements mentioned above in connection with FIGS. 1-3.In practice, portions of process 400 may be performed by differentelements of the described system. It should be appreciated that process400 may include any number of additional or alternative tasks, the tasksshown in FIG. 4 need not be performed in the illustrated order, andprocess 400 may be incorporated into a more comprehensive procedure orprocess having additional functionality not described in detail herein.Moreover, one or more of the tasks shown in FIG. 4 could be omitted froman embodiment of the process 400 as long as the intended overallfunctionality remains intact.

First, the process 400 increases available storage in the data storagenetwork, to create increased available storage (step 402). Increasedavailable storage may include an additional storage either to storemetrics data from a new datacenter or from one of earlier connecteddatacenters. For example, increased available storage may include,without limitation, servers and/or any other system memory elementapplicable to a datacenter and/or compatible with data communication inthe data storage network. Here, the process 400 incorporates additionalstorage resources into the data storage network by creating acommunication connection between the additional storage resources andthe data storage network. This process also includes: (i)reconfiguration of a visualization layer by adding references to newelements of the data storage network, so that the visualization layercan forward users' requests to these new elements of the data storagenetwork; (ii) reconfiguration of one or more data accumulators in theaccessible datacenter by adding references to the new elements of thedata storage network, so these data accumulators can offload accumulateddata to the new elements of the data storage network; and (iii)reconfiguration of existing elements of the data storage network, sothat the existing elements can establish connections to the new elementsof the data storage network.

Next, the process 400 provides access to the increased available storagecontinuously, without rendering the data storage network inaccessibleduring scaling (step 404). Generally, the integration of additionalstorage resources requires a period of “offline” time, during whichadditional memory resources (e.g., a new datacenter, new servers, othermemory hardware) are integrated into the data storage network. Here, theprocess 400 makes data stored at the increased available storage (i.e.,the additional storage resources) available to a user, without takingthe data storage network offline for any period of time.

FIG. 5 is an embodiment of a process 500 for integrating a new securedatacenter into a data storage network. For ease of description andclarity, it is assumed that the process 500 begins by detecting a newdatacenter connected to the data storage network (step 502). Integrationof a new secure data center with the data storage network is a part of aprocess to build out new data centers. The integration is scheduled by aproject plan and must be completed before the data center is availablefor general use.

Next, the process 500 expands a storage layer in an accessibledatacenter connected to the data storage network, by incorporatingadditional storage resources (step 504), such as additional storageservers, thereby increasing the storage capacity of the accessibledatacenter. Here, the process 500 incorporates a data accumulator intoan accessible datacenter, wherein the data accumulator is configured toact as a queue or buffer and receive metrics data from the newdatacenter. Data storage layer expansion includes adding new servers tothe accessible data center, initial setup of the new servers so they canact as data storage “nodes” (i.e., data storage servers). The datastorage network is divided into sub-networks, which are isolated fromeach other. In situations where the new storage servers are intended foraddition and use in one of the existing sub-networks (to expand the datastorage capacity of the existing sub-network), then other servers in thesub-network are also reconfigured by adding references to the newstorage servers.

After expanding the storage layer in the accessible datacenter (step504), the process 500 connects a data pipeline from the new securedatacenter to the storage layer (step 506). Here, the process 500provides a mechanism for communication from the new secure datacenter tothe accessible datacenter. Exemplary embodiments of the process 500 mayimplement this communication mechanism using any compatible datatransmission system, such as Apache Kafka, Rabbit MQ, HTTP RESTend-point, or the like.

In certain embodiments, the process 500 identifies a first dataaccumulator associated with the new secure datacenter, and then connectsthe data pipeline from the first data accumulator to a second dataaccumulator associated with an accessible datacenter, as shown in FIGS.3A-3B. The data accumulator may be implemented using a queue, a buffer,or any region of a physical memory storage used to temporarily store aset of data while the set of data is being moved from one place toanother, as described previously with regard to FIG. 1. Here, theprocess 500 may receive metrics data at the second data accumulator, viathe connected data pipeline, and shift the received data to one or moreof a plurality of storage nodes which are accessible to an end user.

After connecting the data pipeline (step 506), the process 500 providesend user access to the storage layer (step 508). One suitablemethodology for providing end user access to the storage layer isdescribed below with reference to FIG. 6. The storage layer may includea data accumulator in an accessible datacenter, as shown in FIGS. 2-3.The storage layer may also include a plurality of storage nodes,servers, and/or any other compatible form of storage. The process 500provides end user access such that a user may request, retrieve, orotherwise access metrics data transmitted from the new securedatacenter, and which is currently available at the accessibledatacenter.

In certain embodiments, the end user access is provided via avisualization layer, as shown and described above with regard to FIGS.2-3. The visualization layer is generally implemented as a userinterface configured to receive user requests for particular data and todisplay the requested data when retrieved. Here, the process 500 mayreceive user-entered parameters defining a set of data that an end userhas requested, filter all data stored in a particular storage node of anaccessible datacenter to identify the requested data, and present therequested data. User-entered data may include a time period thatincludes the requested metrics data, and a name or identifier for therequested metrics data. Visualization layer includes one or more serversworking together to process requests from many users in parallel. Eachof the servers can process multiple requests at the same time.Processing a user request generally includes: receiving a request froman end user over a communication network, analyzing the request,extracting filters provided by the users to identify metrics the userrequested, identifying servers where metrics satisfying the filters arestored, requesting metrics data from the identified storage servers,performing calculations and transformations on the metrics data if suchwas requested by the user, representing the result metrics data in aformat requested by the user, and sending the result back to the userover a communication network. The visualization layer is alsoresponsible for processing user requests for any elements of the visualuser interface. For example, servers of the visualization layer receiverequests from a user for components that implement elements of a visualuser interface, find such components in a physical memory attached tothe servers, and post the components back the user over a communicationnetwork.

FIG. 6 is an embodiment of a process 600 for providing an end useraccess to secure datacenter metrics data. It should be appreciated thatthe process 600 described in FIG. 6 represents one embodiment of step508 described above in the discussion of FIG. 5, including additionaldetail. First, the process 600 continuously transmits the metrics data,via a data pipeline, to a storage layer of the accessible datacenter(step 602). In this step, the process 600 shifts the metrics data from ahigh-security location to a low-security location, enabling an end userto access the metrics data without delay during integration of thesecure datacenter into a data storage network. In certain embodiments,the process 600 may transmit the metrics data from a first dataaccumulator associated with the secure datacenter to a second dataaccumulator associated with the accessible datacenter (as describedpreviously with regard to FIGS. 1-3). In this particular embodiment, theprocess 600 may shift the metrics data out of the second dataaccumulator, which acts as a buffer or queue, into one of a plurality ofstorage nodes of the storage layer of the accessible datacenter. Inother embodiments, the process 600 may transfer the metrics data fromany secure data storage location associated with the secure datacenterto any less-secure data storage location associated with the accessibledatacenter.

Next, the process 600 receives, via a visualization layer of anaccessible datacenter, an end user request for metrics data associatedwith a secure datacenter (step 604). In certain embodiments, during step604, the secure datacenter may be integrated into the same data storagenetwork as the accessible datacenter. In some embodiments, however, step604 is performed while the secure datacenter is in the process of beingintegrated into the same data storage network as the accessibledatacenter. The requested metrics data may include information relatedto performance of the secure datacenter, as described previously withregard to FIG. 1. The process 600 then permits end user access to thestorage layer, via the visualization layer (step 606), which isdescribed previously with regard to FIGS. 2-3. Here, the process 600provides the end user access to data that continuously flows from securedatacenter(s) to the accessible datacenter. This data is transmittedconstantly, without regard for user requests that may be received or notreceived. Thus, the user is granted access to data that has already beentransmitted to the accessible datacenter.

Techniques and technologies may be described herein in terms offunctional and/or logical block components, and with reference tosymbolic representations of operations, processing tasks, and functionsthat may be performed by various computing components or devices. Suchoperations, tasks, and functions are sometimes referred to as beingcomputer-executed, computerized, software-implemented, orcomputer-implemented. In practice, one or more processor devices cancarry out the described operations, tasks, and functions by manipulatingelectrical signals representing data bits at memory locations in thesystem memory, as well as other processing of signals. The memorylocations where data bits are maintained are physical locations thathave particular electrical, magnetic, optical, or organic propertiescorresponding to the data bits. It should be appreciated that thevarious block components shown in the figures may be realized by anynumber of hardware, software, and/or firmware components configured toperform the specified functions. For example, an embodiment of a systemor a component may employ various integrated circuit components, e.g.,memory elements, digital signal processing elements, logic elements,look-up tables, or the like, which may carry out a variety of functionsunder the control of one or more microprocessors or other controldevices.

When implemented in software or firmware, various elements of thesystems described herein are essentially the code segments orinstructions that perform the various tasks. The program or codesegments can be stored in a processor-readable medium or transmitted bya computer data signal embodied in a carrier wave over a transmissionmedium or communication path. The “computer-readable medium”,“processor-readable medium”, or “machine-readable medium” may includeany medium that can store or transfer information. Examples of theprocessor-readable medium include an electronic circuit, a semiconductormemory device, a ROM, a flash memory, an erasable ROM (EROM), a floppydiskette, a CD-ROM, an optical disk, a hard disk, a fiber optic medium,a radio frequency (RF) link, or the like. The computer data signal mayinclude any signal that can propagate over a transmission medium such aselectronic network channels, optical fibers, air, electromagnetic paths,or RF links. The code segments may be downloaded via computer networkssuch as the Internet, an intranet, a LAN, or the like.

For the sake of brevity, conventional techniques related to signalprocessing, data transmission, signaling, network control, and otherfunctional aspects of the systems (and the individual operatingcomponents of the systems) may not be described in detail herein.Furthermore, the connecting lines shown in the various figures containedherein are intended to represent exemplary functional relationshipsand/or physical couplings between the various elements. It should benoted that many alternative or additional functional relationships orphysical connections may be present in an embodiment of the subjectmatter.

Some of the functional units described in this specification have beenreferred to as “modules” in order to more particularly emphasize theirimplementation independence. For example, functionality referred toherein as a module may be implemented wholly, or partially, as ahardware circuit comprising custom VLSI circuits or gate arrays,off-the-shelf semiconductors such as logic chips, transistors, or otherdiscrete components. A module may also be implemented in programmablehardware devices such as field programmable gate arrays, programmablearray logic, programmable logic devices, or the like. Modules may alsobe implemented in software for execution by various types of processors.An identified module of executable code may, for instance, comprise oneor more physical or logical modules of computer instructions that may,for instance, be organized as an object, procedure, or function.Nevertheless, the executables of an identified module need not bephysically located together, but may comprise disparate instructionsstored in different locations that, when joined logically together,comprise the module and achieve the stated purpose for the module. Amodule of executable code may be a single instruction, or manyinstructions, and may even be distributed over several different codesegments, among different programs, and across several memory devices.Similarly, operational data may be embodied in any suitable form andorganized within any suitable type of data structure. The operationaldata may be collected as a single data set, or may be distributed overdifferent locations including over different storage devices, and mayexist, at least partially, merely as electronic signals on a system ornetwork.

While at least one exemplary embodiment has been presented in theforegoing detailed description, it should be appreciated that a vastnumber of variations exist. It should also be appreciated that theexemplary embodiment or embodiments described herein are not intended tolimit the scope, applicability, or configuration of the claimed subjectmatter in any way. Rather, the foregoing detailed description willprovide those skilled in the art with a convenient road map forimplementing the described embodiment or embodiments. It should beunderstood that various changes can be made in the function andarrangement of elements without departing from the scope defined by theclaims, which includes known equivalents and foreseeable equivalents atthe time of filing this patent application.

What is claimed is:
 1. An accessible datacenter connected to a datastorage network, the accessible datacenter comprising: at least one datapipeline, each of the at least one data pipeline connected to arespective secure datacenter and configured to receive data from eachrespective secure datacenter; a storage layer, configured to store thedata received via the at least one data pipeline; a visualization layer,configured to provide a user interface and to receive user inputrequesting access to the data; and at least one processor, configuredto: detect scaling of the data storage network, the scaling comprisingincreasing available storage of the data storage network, the increasingavailable storage creating increased available storage; and provideaccess to the increased available storage continuously, via thevisualization layer, without rendering the data storage networkinaccessible during scaling; wherein the data is associated with theincreased available storage.
 2. The accessible datacenter of claim 1,wherein the at least one processor is further configured to: detect anew secure datacenter connected to the data storage network; and expandthe storage layer, based on detecting the new secure datacenter; whereinthe increased available storage comprises a storage capacity addition toan existing secure datacenter.
 3. The accessible datacenter of claim 1,wherein the at least one processor is further configured to: detect anew secure datacenter connected to the data storage network; and expandthe storage layer, based on detecting the new secure datacenter; whereinthe increased available storage comprises the new secure datacenter. 4.The accessible datacenter of claim 3, wherein the at least one processoris further configured to connect one of the at least one data pipelineto the new secure datacenter, wherein the one of the at least one datapipeline connects the storage layer to the new secure datacenter; andwherein providing access to the increased available storage furthercomprises providing end user access to the storage layer.
 5. Theaccessible datacenter of claim 3, wherein expanding the storage layerfurther comprises integrating a second data accumulator into theaccessible datacenter, connected to the at least one data pipeline; andwherein the at least one data pipeline is further configured to connecta first data accumulator of the new secure datacenter to the second dataaccumulator of the accessible datacenter.
 6. The accessible datacenterof claim 5, wherein the second data accumulator is configured to receivethe data via the at least one data pipeline.
 7. The accessibledatacenter of claim 6, wherein the data comprises metrics dataassociated with the new secure datacenter.
 8. A processor implementedmethod in an accessible datacenter connected to a data storage network,the accessible datacenter comprising: at least one data pipeline, eachof the at least one data pipeline connected to a respective securedatacenter and configured to receive data from each respective securedatacenter; a storage layer, configured to store the data received viathe at least one data pipeline; and a visualization layer, configured toprovide a user interface and to receive user input requesting access tothe data; and the method comprising: detecting scaling of the datastorage network, the scaling comprising increasing available storage ofthe data storage network, the increasing available storage creatingincreased available storage; and providing access to the increasedavailable storage continuously, via the visualization layer, withoutrendering the data storage network inaccessible during scaling; whereinthe data is associated with the increased available storage.
 9. Themethod of claim 8, further comprising: detecting a new secure datacenterconnected to the data storage network; and expanding the storage layer,based on detecting the new secure datacenter; wherein the increasedavailable storage comprises a storage capacity addition to an existingsecure datacenter.
 10. The method of claim 8, further comprising:detecting a new secure datacenter connected to the data storage network;and expanding the storage layer, based on detecting the new securedatacenter; wherein the increased available storage comprises the newsecure datacenter.
 11. The method of claim 10, further comprisingconnecting one of the at least one data pipeline to the new securedatacenter, wherein the one of the at least one data pipeline connectsthe storage layer to the new secure datacenter; and wherein providingaccess to the increased available storage further comprises providingend user access to the storage layer.
 12. The method of claim 10,wherein expanding the storage layer further comprises integrating asecond data accumulator into the accessible datacenter, connected to theat least one data pipeline; and wherein the at least one data pipelineis further configured to connect a first data accumulator of the newsecure datacenter to the second data accumulator of the accessibledatacenter.
 13. The method of claim 12, wherein the second dataaccumulator is configured to receive the data via the at least one datapipeline.
 14. The method of claim 13, wherein the data comprises metricsdata associated with the new secure datacenter.
 15. An accessibledatacenter connected to a data storage network, the accessibledatacenter comprising: at least one data pipeline, each of the at leastone data pipeline connected to a respective secure datacenter andconfigured to receive data from each respective secure datacenter; astorage layer, configured to store the data received via the at leastone data pipeline; a visualization layer, configured to provide a userinterface and to receive user input requesting access to the data; atleast one processor; and non-transient computer readable media encodedwith programming instructions configurable to cause the at least oneprocessor to: detect scaling of the data storage network, the scalingcomprising increasing available storage of the data storage network, theincreasing available storage creating increased available storage; andprovide access to the increased available storage continuously, via thevisualization layer, without rendering the data storage networkinaccessible during scaling; wherein the data is associated with theincreased available storage.
 16. The accessible datacenter of claim 15,wherein the programming instructions are configurable to cause the atleast one processor to: detect a new secure datacenter connected to thedata storage network; and expand the storage layer, based on detectingthe new secure datacenter; wherein the increased available storagecomprises a storage capacity addition to an existing secure datacenter.17. The accessible datacenter of claim 15, wherein the programminginstructions are configurable to cause the at least one processor to:detect a new secure datacenter connected to the data storage network;and expand the storage layer, based on detecting the new securedatacenter; wherein the increased available storage comprises the newsecure datacenter.
 18. The accessible datacenter of claim 17, whereinthe programming instructions are configurable to cause the at least oneprocessor to connect one of the at least one data pipeline to the newsecure datacenter, wherein the one of the at least one data pipelineconnects the storage layer to the new secure datacenter; and whereinproviding access to the increased available storage further comprisesproviding end user access to the storage layer.
 19. The accessibledatacenter of claim 17, wherein expanding the storage layer furthercomprises integrating a second data accumulator into the accessibledatacenter, connected to the at least one data pipeline; and wherein theat least one data pipeline is further configured to connect a first dataaccumulator of the new secure datacenter to the second data accumulatorof the accessible datacenter.
 20. The accessible datacenter of claim 19,wherein the second data accumulator is configured to receive the datavia the at least one data pipeline and the data comprises metrics dataassociated with the new secure datacenter.